A nonlinear neural network model of mixture of local principal component analysis: application to handwritten digits recognition

Principal component analysis (PCA) is a popular tool in multivariate statistics and pattern recognition. Recently, some mixture models of local principal component analysis have attracted attention due to a number of bene"ts over global PCA. In this paper, we propose a mixture model by concurrently performing global data partition and local linear PCA. The partition is optimal or near optimal, which is realized by a soft competition algorithm called &neural gas'. The local PCA type representation is approximated by a neural learning algorithm in a nonlinear autoencoder network, which is set up on the generalization of the least-squares reconstruction problem leading to the standard PCA. Such a local PCA type representation has a number of numerical advantages, for example, faster convergence and insensitive to local minima. Based on this mixture model, we describe a modular classi"cation scheme to solve the problem of handwritten digits recognition. We use 10 networks (modules) to capture di!erent features in the 10 classes of handwritten digits, with each network being a mixture model of local PCA type representations. When a test digit is presented to all the modules, each module provides a reconstructed pattern by a prescribed principle and the system outputs the class label by comparing the reconstruction errors from the 10 networks. Compared with some traditional neural network-based classi"ers, our scheme converges faster and recognizes with higher accuracy. For a relatively small size of each module, the classi"cation accuracy reaches 98.6% on the training set and 97.8% on the testing set. ( 2000 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.